Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support coroutine functions #346

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

tlambert03
Copy link
Member

@tlambert03 tlambert03 commented Dec 20, 2024

fixes #345

Copy link

codecov bot commented Dec 20, 2024

Codecov Report

Attention: Patch coverage is 68.75000% with 30 lines in your changes missing coverage. Please review.

Project coverage is 98.64%. Comparing base (313192d) to head (037cc17).

Files with missing lines Patch % Lines
src/psygnal/_async.py 66.29% 30 Missing ⚠️

❌ Your patch check has failed because the patch coverage (68.75%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff             @@
##              main     #346      +/-   ##
===========================================
- Coverage   100.00%   98.64%   -1.36%     
===========================================
  Files           21       22       +1     
  Lines         2120     2216      +96     
===========================================
+ Hits          2120     2186      +66     
- Misses           0       30      +30     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@davidbrochart
Copy link

Thanks for the quick PR @tlambert03!
In #345 you mentioned that:

psygnal itself has no concept of an event loop (it's a "bring-your-own-event-loop" library)

But this PR currently assumes that the event loop is asyncio. However Trio and AnyIO are very popular, so I'm wondering how we could support them and really be event-loop agnostic.

@davidbrochart
Copy link

Because structured concurrency always needs a task group in order to launch tasks, I think it's better to not support async callbacks in psygnal. One can just use a sync callback, and have the async logic in the user code where the new value is put synchronously in a stream, and a background task gets the values from the stream asynchronously. That way the user can launch this background task in the appropriate task group.

@tlambert03
Copy link
Member Author

Thanks for your input!

I think it's better to not support async callbacks in psygnal. One can just use a sync callback, and have the async logic in the user code where the new value is put synchronously in a stream, and a background task gets the values from the stream asynchronously.

Are you sure? I'm definitely happy to do whatever we can here to make using coroutines in psygnal easier. That could include supporting auto detection of asyncio, trio, or anyio.

I don't have a ton of personal experience with async/await, so would happily take advice and implement whatever folks suggest.
At the very least we could add some better documentation explaining/showing the strategy you mentioned for using a sync function to put a task in a stream.

And if there are conveniences we could add around that pattern, that's fine too

Comment on lines +20 to +22
@pytest.mark.asyncio
async def test_slot_types(type_: str, capsys: Any) -> None:
backend = _async.set_async_backend("asyncio")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidbrochart I think one of the biggest questions for me is "proper usage". This is what I did to get the tests to work (so far, just for asyncio). But note that I had to allow _async.set_async_backend("asyncio") to be called multiple times within a session... (with the restriction that it be called with the same backend).

An important thing to note there though is that, in set_async_backend, I actually had to let it re-initialize the backend (can't just return the already instantiated backend)... otherwise I get

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

on follow-up tests.

Another question I have here is how to test all three backends. Is it possible within a single python session?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should still prevent set_async_backend() from being called multiple times, unless a clear_async_backend() function is called in between (mostly for tests)?

assert cb.dereference() is not None

cb.cb((2,))
await asyncio.sleep(0.01)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to your comment above... i suspect this is flaky. Should we have an API to wait until the backend has no items in the queue?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a possibility, but a simpler approach could be to set an event in the mock, and wait for it (with a timeout)?

@tlambert03
Copy link
Member Author

one additional note, I'm skipping the _async.py module from the mypyc compilation targets for now. When I include it, it's giving me odd, seemingly-unrelated errors in other modules. it might have to do with trio/anyio missing from the environment during mypyc compilation. But I think it's probably just asking for problems anyway. The key bits (signal emissions and weak-callbacks) are still compiled

Copy link

codspeed-hq bot commented Feb 17, 2025

CodSpeed Performance Report

Merging #346 will not alter performance

Comparing tlambert03:support-async (037cc17) with main (313192d)

Summary

✅ 67 untouched benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Connect signal to async callback
2 participants